AI-driven organizations are using data and machine learning to solve their hardest problems and are reaping the rewards. "Companies that fully absorb AI in their value-producing workflows by 2025 will dominate the 2030 world economy with 120% cash flow growth,"1 according to McKinsey Global Institute. Machine learning (ML) systems have a special capacity for creating technical debt if not managed well. They have all of the maintenance problems of traditional code plus an additional set of ML-specific issues: ML systems have unique hardware and software dependencies, require testing and validation of data as well as code, and as the world changes around us deployed ML models degrade over time. Moreover, ML systems underperform without throwing errors, making identifying and resolving issues especially challenging.

AI-driven organizations are using data and machine learning to solve their hardest problems and are reaping the rewards. "Companies that fully absorb AI in their value-producing workflows by 2025 will dominate the 2030 world economy with 120% cash flow growth,"1 according to McKinsey Global Institute. Machine learning (ML) systems have a special capacity for creating technical debt if not managed well. They have all of the maintenance problems of traditional code plus an additional set of ML-specific issues: ML systems have unique hardware and software dependencies, require testing and validation of data as well as code, and as the world changes around us deployed ML models degrade over time. Moreover, ML systems underperform without throwing errors, making identifying and resolving issues especially challenging.

In a recent blog post, Google announced the beta of Cloud AI Platform Pipelines, which provides users with a way to deploy robust, repeatable machine learning pipelines along with monitoring, auditing, version tracking, and reproducibility. With Cloud AI Pipelines, Google can help organizations adopt the practice of Machine Learning Operations, also known as MLOps – a term for applying DevOps practices to help users automate, manage, and audit ML workflows. Typically, these practices involve data preparation and analysis, training, evaluation, deployment, and more. When you're just prototyping a machine learning (ML) model in a notebook, it can seem fairly straightforward. But when you need to start paying attention to the other pieces required to make an ML workflow sustainable and scalable, things become more complex.

When you're just prototyping a machine learning (ML) model in a notebook, it can seem fairly straightforward. But when you need to start paying attention to the other pieces required to make a ML workflow sustainable and scalable, things become more complex. A machine learning workflow can involve many steps with dependencies on each other, from data preparation and analysis, to training, to evaluation, to deployment, and more. It's hard to compose and track these processes in an ad-hoc manner--for example, in a set of notebooks or scripts--and things like auditing and reproducibility become increasingly problematic. Cloud AI Platform Pipelines provides a way to deploy robust, repeatable machine learning pipelines along with monitoring, auditing, version tracking, and reproducibility, and delivers an enterprise-ready, easy to install, secure execution environment for your ML workflows.

A scalable machine learning workflow involves several steps and complex computations. These steps include data preparation and preprocessing, training and evaluating models, deploying these models and much more. While prototyping a machine learning model can be seen as a simple and easygoing task, it eventually becomes hard to track each and every process in an ad-hoc manner. To simplify the development of machine learning models, Google launches the beta version of Cloud AI Platform Pipelines, which will help to deploy robust, repeatable machine learning pipelines along with monitoring, auditing, version tracking, and reproducibility. It ensures to deliver an enterprise-ready, easy to install, a secure execution environment for the machine learning workflows. The AI platform in Google Cloud is a code-based data science development environment, which helps the machine learning developers, data scientists and data engineers to deploy ML models in a quick and cost-effective manner.

The Google Cloud team just celebrated the beta launch of its AI Platform Pipelines feature with a couple of additions and improvements to the machine learning workflow execution environment. The product was started to provide those at the beginning of their machine learning journey with a way to "deploy robust, repeatable machine learning pipelines along with monitoring, auditing, version tracking, and reproducibility" in an "easy to install, secure" environment. It is therefore mainly made up of the infrastructural component needed to run the workflows, as well as tools for creating and sharing pipelines. Since the service is part of Google Cloud, it can be quickly installed via the company's cloud console, which also takes care of access management. Options for the building pipelines part boil down to the Kubeflow Pipelines SDK, which isn't surprising given that the AI Platform Pipelines run on a GKE cluster, and the development kit for TensorFlow Extended (TFX), TF's end-to-end machine learning platform.

Google today announced the beta launch of Cloud AI Platform Pipelines, a service designed to deploy robust, repeatable AI pipelines along with monitoring, auditing, version tracking, and reproducibility in the cloud. Google's pitching it as a way to deliver an "easy to install" secure execution environment for machine learning workflows, which could reduce the amount of time enterprises spend bringing products to production. "When you're just prototyping a machine learning model in a notebook, it can seem fairly straightforward. But when you need to start paying attention to the other pieces required to make a [machine learning] workflow sustainable and scalable, things become more complex," wrote Google product manager Anusha Ramesh and staff developer advocate Amy Unruh in a blog post. "A machine learning workflow can involve many steps with dependencies on each other, from data preparation and analysis, to training, to evaluation, to deployment, and more. It's hard to compose and track these processes in an ad-hoc manner -- for example, in a set of notebooks or scripts -- and things like auditing and reproducibility become increasingly problematic."